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1.  Introduction 


The  U.S.  Army  is  currently  developing  future  manned  ground  vehicles  that  will  support  indirect- 
vision  driving  (IVD),  that  is,  driving  via  a  visual  display  rather  than  direct  viewing  of  the 
environment.  The  current  plans  for  the  future  vehicular  programs  are  to  use  a  360°  horizontal/ 
90°  vertical  two-dimensional  (2-D)  display  to  provide  local  situational  awareness  infonnation  to 
the  driver.  However,  recent  field  tests  of  vehicles  using  these  IVD  displays  showed  that  drivers 
tended  to  drive  significantly  more  slowly  than  when  driving  with  normal  direct  view  of  the 
environment,  showing  as  much  as  a  60%  increase  in  time  on  their  driving  tasks  (Flascher,  2008; 
GDLS,  2007).  It  was  reported  that  driving  performance  decrement  may  have  been  linked  to  the 
drivers’  poor  depth  perception  of  the  environment  due  to  the  loss  of  binocular  depth  perception 
when  using  the  2-D  IVD  video  displays,  resulting  in  their  degraded  judgment  of  the  terrain 
features  (GDLS,  2007). 

One  proposed  solution  is  to  investigate  the  effectiveness  of  stereoscopic  displays  (SDs)  for 
improving  driving  performance  in  general,  and  driving  speed  in  particular.  Past  research  has 
shown  that  SDs,  which  rely  on  various  techniques  to  present  binocular  images  to  the  user,  appear 
to  provide  advantages  over  2-D/monoscopic  displays,  such  as  faster  and  more  accurate 
perception  of  the  environment,  better  distance  estimation,  enhanced  detection  of  terrain  hazards 
(e.g.,  slopes  and  depressions),  enhanced  object  recognition  and  detection,  and  visual  noise 
filtering  (Drasic,  1991;  Merritt,  CuQlock-Knopp,  and  Myles,  1997;  Scribner  and  Gombash, 

1998;  Singer,  Ehrlich,  Cinq-Mars,  and  Papin,  1995).  Studies  have  shown  that  SDs  could 
enhance  telerobotic  vehicle  operators’  speed  of  navigation  and  distance  judgment  (Holzhausen, 
Pitrella,  and  Wolf,  1993;  Spain  and  Hughes,  1991;  Umeda,  Martin,  and  Merritt,  1991). 

1.1  Stereoscopic  Displays 

Stereoscopic  displays  can  provide  many  advantages  over  traditional  monocular  displays  in 
driving,  particularly  in  off-road,  military  driving.  According  to  Dumbreck,  Smith,  and  Murphy 
(as  cited  in  Drascic,  1991),  remote  manipulation  tasks  that  involve  “ballistic  movement, 
recognition  of  unfamiliar  scenes,  analysis  of  three  dimensionally  complex  scenes  and  the 
accurate  placement  of  manipulators  or  tools  within  such  scenes”  especially  benefit  from  SDs. 
However,  empirical  studies  examining  the  utility  of  SDs  generally  report  that  SDs  might  be 
useful  in  only  certain  circumstances.  For  example,  Drascic  (1991)  found  that  the  benefits  of 
SDs,  while  longer  lasting  for  tasks  that  required  binocular  depth  cues  (i.e.,  using  a  robot  to  place 
an  object  between  two  “bombs”  separated  by  8  cm),  did  not  last  as  long  for  tasks  that  did  not 
require  much  binocular  depth  perception  (i.e.,  same  task  with  “bombs”  separated  by  64  cm). 
Generally,  participants  quickly  learned  how  to  use  the  monocular  cues  available  in  the 
monocular  displays  to  accomplish  those  tasks. 
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Draper,  Handel,  and  Hood  (1991)  had  participants  perform  Fitts  Law  (Fitts,  1954)  tapping  tasks 
and  reported  that  SDs  were  only  useful  for  more  difficult  tasks  and  only  for  inexperienced 
participants.  They  suggested  that  SDs  would  be  useful  when  the  image  quality,  task  structure 
and  predictability,  user  experience,  and  manipulator  dexterity  were  suboptimal.  Rosenberg 
(1993)  found  that  SDs  helped  depth-matching  perfonnance  and  the  distances  between  the  two 
cameras  affected  the  usefulness  of  the  SDs.  They  reported  that  the  best  performance  was 
achieved  when  the  inter-camera  distance  was  less  than  the  interocular  distance  (i.e.,  2-3  cm  vs. 

6  cm).  On  the  other  hand,  Green,  Dougherty,  and  Savacool  (2003)  did  not  find  significant 
benefits  of  using  SDs  for  their  task,  teleoperating  shipboard  cranes  to  place  cargo,  in  terms  of 
time  and  accuracy  of  task  performance  and  depth  perception.  As  for  user  preference,  a 
consistent  finding  from  various  studies  is  that  teleoperators  generally  prefer  SDs  over  monocular 
displays  (Drascic  and  Grodski,  1993;  Green  Dougherty,  and  Savacool,  2003).  However,  as  noted 
in  Scribner  and  Gombash  (1998),  artificially  induced  binocular  stereovision  may  increase  motion 
sickness  and  perceived  stress. 

More  realistic  tests  were  conducted  by  researchers  from  the  U.S.  Army  who  investigated  the 
ability  of  humans  to  detect  obstacles  in  static  and  moving  video  terrain  with  three-dimensional 
(3-D)  and  hyperstereo  (ocular  distance  artificially  increased  to  accentuate  depth  cues)  displays. 
The  results  indicated  improved  detection  of  negative  terrain  and  mobility  obstacles  for  the  3-D 
conditions  vs.  2-D  conditions  (Merritt,  CuQlock-Knopp,  Kregel,  Smoot,  and  Monaco,  2005; 
Merritt,  CuQlock-Knopp,  and  Myles,  1997).  Preliminary  field  demonstrations  of  actual  Army 
systems  suggest  the  3-D  perfonnance  gains  will  extend  to  teleoperated  systems.  The  researchers 
caution  that  SDs  have  definite  perceptual  and  physical  limitations,  as  mentioned  above;  however, 
SDs  should  be  an  optional  mode  for  complex  terrain,  especially  where  depth  perception  is 
crucial,  and  for  arm  manipulations  and  other  tasks  where  nonnal  3-D  cues  are  unavailable 
(B.  Vaughan,  personal  communication,  16  March  2006). 

1.2  Current  Study 

In  the  current  study,  we  investigated  whether  an  SD  could  improve  the  operators’  IVD  and  robot 
teleoperation  performance,  and  which  type  of  SD  supported  better  performance.  We  evaluated 
two  types  of  SD  technologies:  active  shutter  glasses  (nVIDIA*)  and  passive  polarized  glasses 
(Miracubc+).  The  experiment  had  three  parts  -  Perceptual  evaluation,  Robot  Teleoperation,  and 
Virtual  (simulated  IVD).  For  the  Perceptual  part  of  the  experiment,  participants  viewed  still 
images  of  terrain  using  one  of  the  SDs  in  3-D  and  responded  about  object  distance  from  the 
camera  as  well  as  object  height  on  elevation  planes.  They  also  viewed  videos  of  hazardous 
terrain  using  one  of  the  SDs  in  3-D  and  responded  when  they  detected  a  terrain  drop-off.  For  the 
Robot  Teleoperation  part  of  the  experiment,  participants  maneuvered  a  robot  through  a  course  of 
cones  on  a  grass  terrain  using  one  of  the  SDs  in  both  2-D  and  3-D  modes.  For  the  Virtual  part  of 


* 

nVIDIA  is  a  registered  trademark  of  nVIDIA  Corp. 
^Miracube  is  a  registered  trademark  of  Miracube,  Inc. 
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the  experiment,  participants  drove  through  both  3-D  stereo  and  non-stereo  scenarios  in  a  3-D 
rendered,  simulated  driving  environment  (VBS2*).  Different  types  of  terrains  were  simulated  in 
the  virtual  scenarios. 


2.  Method 


2.1  Participants 

Thirty-two  individuals  (19  males  and  13  females,  mean  age  =  25.8)  from  the  Orlando,  FL  area 
participated  in  the  experiment.  Participants  received  payment  for  their  time  at  the  rate  of 
$  1 5/hour. 

2.2  Apparatus 

2.2.1  Stereoscopic  Display  Systems 

The  stereoscopic  display  systems  used  were  an  nVIDIA  and  a  Pavonine  Miracube  display 
(figure  1). 


Figure  1 .  Stereoscopic  displays  used  in  the  virtual  tests. 


*  . 

Virtual  Battlespace  2  (VSB2)  is  a  registered  trademark  of  Bohemia  Interactive  Australia  (BIA). 
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The  nVIDIA  system  used  a  22-in  120  Hz  display,  with  a  resolution  of  1680  x  1050  pixels, 
viewed  with  active  3-D  shutter  glasses  synchronized  by  an  infrared  (IR)  emitter  pointing  toward 
the  glasses.  The  stereoscopic  video  is  sent  as  an  alternating  series  of  left  eye  and  right  eye 
frames.  Therefore,  the  frame  rate  perceived  by  the  user  is  60  Hz.  All  frames  have  full  pixel 
resolution.  To  achieve  proper  depth,  convergence,  and  orthostereo  (the  ability  to  produce  images 
in  the  display  that  are  the  same  visual  size  as  in  the  real  world),  the  display  had  to  be  calibrated 
each  time  the  experimental  condition  changed  between  2-D  and  3-D.  To  achieve  this,  the  depth 
was  set,  using  the  scroll  bar  on  the  back  of  the  emitter,  to  a  predefined  level  (3/50  for  2-D,  30/50 
for  3-D)  and  the  convergence  was  set,  using  Ctrl+F5  or  Ctrl+F6  on  the  keyboard,  to  a  predefined 
level  (0  for  2-D,  -2.5  in  for  3-D). 

The  Pavonine  system  used  a  24-in  Miracube  display,  with  a  resolution  of  1920  x  1200  pixels. 

The  Miracube  display  uses  a  filter  sheet  placed  in  front  of  the  60  Hz  liquid  crystal  display.  The 
filter  has  alternating  lines  of  polarization  causing  the  odd  pixel  rows  of  the  display  to  be 
polarized  oppositely  from  the  even  rows.  Coupled  with  passive  polarized  3-D  glasses,  this 
provides  the  viewer  with  a  stereoscopic  signal  at  nearly  full  brightness  but  with  half 
vertical  resolution  per  eye.  To  achieve  proper  depth,  convergence,  and  orthostereo,  the  display 
had  to  be  calibrated  each  time  the  experimental  condition  changed  between  2D  and  3D.  To 
achieve  this,  values  were  set  to  a  predefined  level  for  3D  (convergence  =  -0.0306,  separation 
=  562.17%),  and  they  were  toggled  on  and  off  using  the  “?”  key  on  the  keyboard. 

2.2. 1.1  Stereoscopic  Video  Data  Processing.  The  video  data  path  was  as  follows.  Each 
camera  was  connected  to  its  own  dedicated  1394.b  port.  Using  the  synchronized  capture  feature 
of  the  cameras,  frames  are  captured  synchronously  (measured  to  be  <1/480  of  a  second 
difference).  The  two  frame  streams  are  then  combined  by  the  Stereoscopic  Multiplexer* 
software  program  (for  more  information,  please  see  ).  The 

Multiplexer  provides  two  functions.  First  it  compares  the  time  stamps  of  frames  on  both 
incoming  streams.  If  the  frames  are  not  in  sync,  it  will  drop  frames  until  the  streams  are  in  sync. 
The  Multiplexer  then  merges  the  two  streams  into  a  single,  double-wide  side-by-side  2048  x  768 
video  stream,  which  is  then  encoded  using  xvid  compression,  packetized,  and  sent  over  the 
network  using  two  bridged  Linksys’1'  WRT600N  routers.  The  receiving  computer  then 
depacketizes  the  stream  and  makes  it  available  as  a  virtual  live  camera  to  the  system.  Any 
Windows  direct-show  compatible  device  can  then  connect  to  the  virtual  camera  and  receive  the 
video  stream.  Stereoscopic  Player®  is  then  used  to  connect  to  the  virtual  camera,  convert  the 
video  into  the  correct  format  for  the  display,  and  play  the  video. 


* 

Stereoscopic  Multiplexer  is  a  registered  trademark  of  3dtv.at. 
'  Linksys  is  a  registered  trademark  of  Cisco  Systems,  Inc. 
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2.2.2  Stereoscopic  Images,  Video,  and  Simulation 

Twenty  still  pictures  were  used  in  the  experiment.  For  ten  of  the  pictures,  participants  were 
asked  to  choose  which  of  two  objects  was  closest  to  the  camera.  For  the  other  ten  pictures, 
participants  were  asked  to  choose  between  two  planes  of  elevation  and  identify  the  object  that 
had  a  higher  elevation.  Ten  pre-recorded  videos  of  terrain  scenes  were  also  used  in  the 
experiment,  each  scene  averaging  30  to  45  s  long.  The  10  scenes  were  of  wooded  areas  and  were 
collected  both  locally  and  remotely  by  a  consultant  (Merritt  et  ah,  2005).  Participants  were 
asked  to  respond  as  soon  as  they  saw  a  terrain  drop-off  in  the  scene. 

Bohemia  Interactive’s  VBS2  (U.S.  Anny  version)  and  the  VBS2  VTK  Developer  Suite  vl.23 
were  used  to  create  the  virtual  driving  simulation.  Custom  3-D  models  that  were  needed  to 
create  the  virtual  scenarios  were  modeled  in  3ds  Max9*.  Custom  textures  were  created  with 
Adobe  '  Photoshop*  CS3.  The  models  and  textures  were  then  converted  to  VBS2  native  formats 
(.paa  and  .p3d)  using  Oxygen2  and  the  Tex  Viewer.  All  terrain  models  larger  than  50  m  had  to 
be  imported  into  VBS2  as  separate  models  and  then  reassembled  to  appear  seamless. 

For  the  Negative  Terrain  tracks,  it  was  important  to  minimize  the  visual  cues  that  are  typically 
present  in  realistically-rendered  2-D  simulations  (e.g.,  high  contrast  lighting,  realistic  textures 
which  give  the  illusion  of  added  depth).  The  Negative  Terrain  tracks  incorporated  both  positive 
terrain  features,  such  as  trees  and  rocks,  and  negative  terrain  features,  such  as  cliffs  and  ditches. 
An  attempt  was  made  to  eliminate  all  visual  hints,  such  as  changes  in  texture  type  or  shadows 
that  would  be  a  cue  to  the  participant  that  a  drop-off  in  elevation  was  ahead.  Flatly-rendered 
textures  with  random  repeating  patterns  (e.g.,  sand  with  weeds  and  plain  grass)  were  used  to 
eliminate  visual  cues  caused  by  the  textures.  A  script  was  created  to  change  the  time  of  day  in 
VBS2  to  dusk  to  eliminate  high  contrast  lighting  (e.g.,  sharp  highlights  and  shadows  in  negative 
terrain  areas). 

VBS2  is  a  very  robust  3-D  game  engine  that  can  render  very  high  polygon  environments,  high 
resolution  textures,  and  many  dynamic  and  static  objects  in  real-time.  Due  to  the  unique 
requirements  of  the  Negative  Terrain  condition,  these  tracks  featured  a  higher  polygon  count  and 
higher  resolution  textures  than  the  other  virtual  conditions.  Computer  performance  varies  in  3-D 
simulations  depending  on  the  polygon  count  and  the  size  and  number  of  textures  being  rendered 
in  real-time.  A  drop  in  computer  perfonnance  is  magnified  to  a  greater  extent  when  the 
environment  is  being  rendered  in  3-D  stereo  in  real-time,  since  the  3-D  engine  must  render  two 
images  per  frame  instead  of  one.  A  slight  drop  in  hardware  performance  was  discovered  on  the 
Negative  Terrain  tracks  due  to  the  reasons  above.  When  in  3-D  stereo  mode,  at  times  the  vehicle 
appeared  to  move  slightly  more  slowly  than  when  in  2-D  mode  while  maintaining  smooth 
driving  conditions  with  no  lag. 

* 


3ds  Max9  is  a  registered  trademark  of  Autodesk,  Inc. 

1  Adobe  is  a  registered  trademark  of  Adobe  Systems,  Inc. 

'  Photoshop  is  a  registered  trademark  of  Adobe  Systems,  Inc. 
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2.2.3  Talon  Robot 


The  small  unmanned  ground  vehicle  (SUGV)  was  a  TALON*  robot  (figure  2),  which  was 
controlled  remotely.  The  TALON  used  in  the  experiment  was  the  small,  mobile  version.  It  has 
been  used  in  military  operations  since  2000,  and  armed  TALONs  began  to  serve  as  battle 
buddies  for  Soldiers  in  2005.  The  two  cameras  used  in  the  experiment,  Sony!  XCD-SX90CR 
cameras  with  a  resolution  of  1024  x  768  pixels,  were  mounted  on  the  main  platform  of  the 
TALON,  with  an  inter-camera  distance  of  2.25  in,  along  with  other  processing  equipment  for  the 
SD  systems.  The  combined  field  of  view  for  the  cameras  was  44.5°.  The  distance  between  the 
cameras  and  the  ground  was  30.5  in. 


Figure  2.  TALON  robot  with  stereoscopic  cameras. 


Talon  is  a  registered  trademark  of  Foster-Miller,  Inc. 
^  Sony  is  a  registered  trademark  of  Sony  Corp. 
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2.2.4  Surveys  and  Tests 

A  demographics  questionnaire  (appendix  A)  was  administered  at  the  beginning  of  the  training 
session.  Participants’  perceived  workload  was  evaluated  using  the  computerized  version  of  the 
National  Aeronautics  and  Space  Administration-Task  Load  Index  (NASA-TLX)  questionnaire 
(appendix  B),  which  includes  a  pairwise  comparison  weighting  procedure  (Hart  and  Staveland, 
1988).  The  NASA-TLX  is  a  self-reported  questionnaire  of  perceived  demands  in  six  areas: 
mental,  physical,  temporal,  effort  (mental  and  physical),  frustration,  and  performance. 
Participants  evaluated  their  perceived  workload  level  in  these  areas  on  10-point  scales  as  well  as 
completed  pairwise  comparisons  for  each  subscale. 

The  Simulator  Sickness  Questionnaire  (SSQ;  see  appendix  C)  was  used  to  evaluate  participants 
simulator  sickness  symptoms  (Kennedy,  Lane,  Berbaum,  and  Lilienthal,  1993).  The  SSQ 
consists  of  a  checklist  of  16  symptoms.  Each  symptom  is  related  in  terms  of  degrees  of  severity 
(none,  slight,  moderate,  severe).  A  total  severity  score  can  be  derived  by  a  weighted  scoring 
procedure  (see  appendix  D)  and  reflects  overall  discomfort  level. 

2.2.5  Stereoscopic  Test 

The  stereoscopic  test  used  was  a  Randot  Stereo  Test*  that  had  four  parts,  three  of  which  were 
used  for  this  experiment.  The  stereo  test  uses  polarized  glasses  worn  by  the  participant  and  is 
held  perpendicular  to  the  ground  16  in  from  the  participant.  The  first  part  of  the  test  was 
identification  of  shapes  in  four  boxes  that  required  a  stereoacuity  of  500  s  of  arc,  and  this  was  the 
point  at  which  participants  were  either  included  or  excluded  from  the  study.  The  second  part  of 
the  test  was  identification  of  shapes  in  four  boxes  that  required  a  stereoacuity  of  250  s  of  arc. 

The  third  part  of  the  test  was  identification  of  which  of  three  circles  stood  out  to  the  participant, 
and  these  required  varying  levels  of  stereoacuity  from  200  to  20  s  of  arc.  Participants’  scores 
(i.e.,  seconds  of  arc)  were  used  as  a  covariate,  stereoacuity,  for  the  analyses. 

2.3  Experimental  Design 

The  overall  design  of  the  study  was  a  2  x  2  mixed-subjects  design.  The  between-subject  factor 
was  Display  type  (nVIDIA  v  Miracube)  and  the  within-subject  factor  was  Stereovision  (2-D  v 
3-D).  However,  for  the  first  part  of  the  experiment,  participants  did  not  evaluate  the  pictures  and 
videos  in  2-D,  as  Merritt  et  al.  (2005)  have  demonstrated  that  terrain  hazards  could  be  detected 
sooner  with  a  3-D  display  than  with  a  2-D  display. 

2.4  Procedure 

After  being  briefed  on  the  purpose  of  the  study  and  signing  an  informed  consent  form, 
participants  completed  the  demographics  questionnaire.  The  experimenter  explained  the 
experiment  to  participants  and  answered  any  questions  they  had  at  the  time.  Not  all  participants 


* 

Randot  stereo  tests  are  produced  by  Precision  Vision. 
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completed  the  three  parts  of  the  experiment  sequentially  due  to  equipment  delays.  The 
participants  then  went  through  the  calibration  process  as  described  in  section  2.2. 1 .  The  distance 
between  the  participant  and  the  SD  was  23  in  for  nVIDIA  and  25  in  for  Miracube. 

2.4.1  Perceptual  Tests 

For  the  Perceptual  tests,  participants  viewed  20  still  pictures;  for  10  of  the  pictures  they 
identified  the  closer  object,  and  for  the  other  10  they  identified  which  object  was  on  a  higher 
elevation  plane.  Participants  then  watched  10  pre-recorded  video  clips  of  hazardous  terrain  in 
3-D  (with  his/her  assigned  SD  system)  and  responded  as  soon  as  they  detected  a  drop-off  in 
terrain.  Each  video  was  -30-45  s  long,  and  after  completing  this  portion,  participants  assessed 
their  workload  and  sickness  symptoms. 

2.4.2  Robot  Teleoperation  Tests 

Participants  practiced  teleoperating  the  robot  (without  line-of-sight)  for  one  course  in  2-D  and 
one  course  in  3-D.  They  were  then  asked  to  drive  a  robot  through  four  courses  marked  by  traffic 
cones  (figure  3),  twice  with  3-D  and  twice  with  2-D  (the  same  display  with  3-D  turned  off).  If 
participants  drove  the  robot  completely  out  of  the  course  (i.e.,  the  entire  robot  was  outside  the 
course),  the  trial  was  restarted,  and  each  participant  was  allowed  to  restart  each  trial  twice.  If 
participants  went  outside  the  course  more  than  three  times  (the  original  trial  plus  two  restarts), 
they  were  moved  to  the  next  course.  The  performance  data  from  the  last  successful  trial  was 
used  with  a  restart  value  reflecting  that  participants  had  to  be  restarted.  Trials  were  counterbalanced 
(Williams  Square)  to  avoid  order  effects.  Each  trial  took  approximately  one  minute  and  after 
completing  this  portion,  participants  assessed  their  workload  and  sickness  symptoms. 


Figure  3.  Robot  teleoperation  tests. 
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2.4.3  Virtual  Tests 

Participants  were  asked  to  drive  through  three  types  of  courses,  and  each  type  of  course  featured 
two  unique  tracks  (figure  4).  One  type  of  course  was  a  “Floating  Objects  Course”  in  which 
participants  drove  on  an  enclosed,  pavement  course  toward  six  sets  of  object  pairs  (figure  4a). 
Participants  were  instructed  to  drive  as  quickly  as  possible  around  the  objects  on  the  side  (left  or 
right)  of  the  closer  object.  Another  type  of  course  was  an  “Obstacle  Course,”  in  which 
participants  drove  as  quickly  as  possible  around  obstacles,  such  as  rocks  and  shrubs  (figure  4b). 
The  third  type  of  course  was  a  “Negative  Terrain  Course”  in  which  participants  drove  on  an 
enclosed  course  of  varying  negative  and  positive  terrain  features  (i.e.,  holes  in  the  ground,  drop- 
offs,  and  hills;  figure  4c).  Participants  were  instructed  to  drive  as  quickly  as  possible  through  the 
course  while  avoiding  the  positive  and  negative  terrain  obstacles.  Each  participant  was  given  a 
practice  course  for  each  type  of  trial  in  2-D  and  3-D  prior  to  starting  the  experimental  trials,  and 
each  participant  completed  each  type  of  experimental  course  twice,  once  in  2-D  and  once  in  3-D. 
Trials  were  counterbalanced  (Williams  Square)  to  avoid  order  effects.  Each  trial  took 
approximately  three  minutes  and  after  completing  all  trials,  participants  assessed  their  workload 
and  sickness  symptoms. 

Following  completion  of  the  three  parts  of  the  experiment,  participants  were  fully  debriefed,  and 
their  information  was  taken  for  payment.  The  entire  experiment  lasted  ~1  1/2  h. 

2.5  Dependent  Measures 

The  dependent  measures  for  the  Perceptual  tests  of  the  experiment  were  response  time  and 
accuracy.  The  dependent  measures  for  the  Robot  Teleoperation  tests  were  course  completion 
time  and  number  of  cones  hit.  The  dependent  measures  for  the  Virtual  tests  were  course 
completion  time  and  accuracy  of  completing  the  simulated  driving  tasks  (i.e.,  how  many  terrain 
hazards  they  failed  to  avoid).  Subjective  measures  included  participants’  assessments  of  their 
workload  (i.e.,  NASA-TLX  scores)  and  sickness  (i.e.,  SSQ)  scores,  which  were  assessed  after 
each  part  of  the  experiment.  Mixed-design  Analyses  of  Covariance  (ANCOVAs)  with  Display 
(nVIDIA  v  Miracube)  as  the  between-subject  factor  and  Stereovision  (2-D  v  3-D)  as  the  within- 
subject  factor  were  used  to  evaluate  the  performance  measures.  Participants’  stereovision  was 
used  as  the  covariate.  All  tests  were  perfonned  using  SPSS  18.0  with  an  alpha  level  of  0.05. 
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Figure  4.  (a)  Floating  objects  course;  (b)  obstacle  course;  and  (c)  negative 
terrain  course. 
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3.  Results 


3.1  Overall  Effects  of  Stereovision  and  Display  Type 

Table  1  lists  several  measures  relating  to  participants’  perfonnance.  The  effects  of  Stereovision 
on  course  completion  times  for  the  Robot  Teleoperation  and  the  three  Virtual  tests  were 
evaluated.  The  analysis  revealed  that  there  was  a  significant  main  effect  for  Stereovision, 
^(4,26)  =  5.53,  p  <  0.005,  with  2-D  being  longer/slower  than  3-D.  The  effect  of  Display  in  the 
3-D  conditions  was  investigated  separately  to  examine  the  effect  of  different  display  types  on 
times  (i.e.,  response  times  for  Video  viewing,  course  completion  times  for  Robot  Teleoperation 
and  the  Virtual  tests).  The  analysis  showed  that  there  was  a  significant  main  effect  for  Display, 
^(1,29)  =  4.46,  p  <  0.05,  with  nVIDIA  being  better  (i.e.,  faster)  than  Miracube.  The  following 
segments  report  the  results  of  each  task. 


Table  1.  Summary  of  performance  measure  means  and  standard  deviations. 


Measures 

2-D 

3-D 

nVIDIA 

Miracube 

nVIDIA 

Miracube 

Perceptual 

Stills 

Response  time  (s) 

— 

— 

8.14 

(6.35) 

6.43 

(5.14) 

Percent  correct 

— 

— 

73 

(44) 

78 

(42) 

Video 

Response  time  (s) 

— 

— 

8.25 

(6.21) 

11.22 

(4.21) 

Robot 

Teleoperation 

— 

Completion  time  (s) 

65.03 

(14.71) 

74.78 

(20.52) 

62.63 

(14.54) 

67.94 

(19.87) 

No.  of  cones  hit 

3.88 

(1.69) 

4.56 

(1.79) 

3.38 

(1.40) 

4.19 

(1.31) 

Virtual 

Floating 

Completion  time  (s) 

15.01 

(3.84) 

15.66 

(4.66) 

16.74 

(5.13) 

17.43 

(6.07) 

No.  of  trials  correct 

3.69 

(i.oi) 

4.44 

(1.03) 

4.44 

(1.21) 

5.19 

(1.11) 

Obstacle 

Completion  time  (s) 

152.6 

(33.4) 

149.2 

(30.4) 

160.3 

(41.0) 

157.7 

(40.2) 

Time  off  course  (s) 

16.73 

(13.52) 

16.92 

(10.14) 

14.84 

(14.09) 

13.85\(9.6 

9) 

Negative 

Completion  time  (s) 

209.8 

(48.7) 

217.6 

(50.0) 

189.2 

(45.0) 

217.2 

(47.7) 

Time  off  course  (s) 

17.44 

(14.29) 

10.11 

(13.93) 

11.23 

(6.12) 

11.13 

(11.39) 

Note:  Standard  deviations  are  presented  in  parentheses. 
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3.2  Perceptual  Tests 


For  the  still  images,  the  Closer  task  and  the  Elevation  task  were  merged  for  the  analyses. 
Additionally,  trials  that  less  than  60%  of  participants  got  correct  were  not  included  in  the 
analyses,  as  that  perfonnance  level  was  just  higher  than  chance  level.  The  effect  of  Display  on 
participants’  response  times  and  accuracy  (number  correct  out  of  8  trials)  was  analyzed.  Display 
did  not  have  a  significant  effect  on  either  response  times  or  accuracy.  For  the  Video  viewing 
part  of  the  tests,  the  effect  of  Display  on  response  times  for  the  10  video  clips  was  analyzed.  The 
analysis  revealed  that  there  was  a  significant  main  effect  for  Display,  F(l,26)  =  5.47,/?  <  0.05, 
with  nVIDIA  (M=  7.33  sec)  being  better  (i.e.,  faster)  than  Miracube  (M=  1 1.87  s)  (figure  5). 


Figure  5.  Response  times  (s)  for  the  perceptual  tests. 

3.3  Robot  Teleoperation 

The  effects  of  Display  and  Stereovision  on  participants’  course  completion  times  and  number  of 
cones  hit  were  analyzed.  There  was  a  marginally  significant  main  effect  of  Display  for  course 
completion  times,  F(l,28)  =  3.024,/?  =  0.093,  with  the  nVIDIA  display  (M=  62.89  s)  resulting 
in  faster  completion  times  than  the  Miracube  display  (M=  73.59  s)  by  more  than  10  s,  which  is 
an  ~15%  reduction  in  completion  times.  While  the  difference  in  completion  times  between 
stereo  conditions  failed  to  reach  statistical  significance,  the  3-D  condition  (M=  65.8  s)  produced 
faster  operating  times  than  the  2-D  condition  (M=  70.6  s)  by  almost  5  s  (figure  6).  In  terms  of 
accuracy  (number  of  cones  hit),  participants  hit  less  cones  in  3-D  and  with  the  nVIDIA  display. 
However,  the  differences  failed  to  reach  statistical  significance. 
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Figure  6.  Robot  teleoperation  completion  times. 

3.4  Virtual  Scenarios 
3.4.1  Floating  Object  Course 

The  effects  of  Display  and  Stereovision  on  participants’  course  completion  times  and  accuracy 
(i.e.,  number  of  trials  correct)  were  analyzed.  For  course  completion  times,  there  was  a 
marginally  significant  interaction  effect  of  Display  x  Stereoacuity,  F(  1,29)  =  3.65,  p  =  0.066, 
indicating  that  completion  times  were  more  affected  by  Stereoacuity  in  the  Miracube  than  the 
nVIDIA  display  (the  Miracube  display  had  a  greater  disparity  in  completion  time  over  trials  than 
the  nVIDIA  display)  and  that  completion  times  were  faster  for  those  who  had  a  better 
stereoacuity  (69  or  below).  Those  participants  using  nVIDIA  who  had  better  stereoacuity  were 
~1  s  faster  than  those  with  worse  stereoacuity;  those  using  Miracube  who  had  better  stereoacuity 
and  were  ~4  s  faster  than  those  with  worse  stereoacuity.  For  accuracy,  3-D  (M=  4.81)  provided 
significantly  more  accurate  results  than  2-D  (M=  4.06),  F(  1 ,29)  =  4.37,/?  <  0.05.  Additionally, 
the  Miracube  display  (M=  4.79)  provided  significantly  more  accurate  results  than  the  nVIDIA 
display  (M=  4.09),  F(l,29)  =  5.00,/?  <  0.05.  Participants’  average  speed  (miles  per  hour)  for  the 
virtual  tests  in  the  3-D  condition  is  depicted  in  figure  7. 
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Figure  7.  Average  speed  (miles  per  hour)  for  the  virtual  tests  in  3-D  condition. 


3.4.2  Obstacle  Course 

The  effects  of  Display  and  Stereovision  on  participants’  course  completion  times  and  accuracy 
(i.e.,  time  off  course)  were  analyzed.  For  accuracy,  there  was  a  marginally  significant  effect  of 
Stereovision  in  that  3-D  (M=  14.15  s)  produced  more  accurate  performance  than  2-D 
(M=  16.25  s),  7^(1 ,27)  =  3.59,  p  =  0.069.  The  percentage  of  time  participants’  were  off-course 
for  the  obstacle  and  the  negative  terrain  courses  are  depicted  in  figure  8. 


Figure  8.  Percentage  of  time  off-course  for  the  obstacle  and  negative  terrain  courses. 
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3.4.3  Negative  Terrain  Course 

The  effects  of  Display  and  Stereovision  on  participants’  course  completion  times  and  accuracy 
(i.e.,  time  off-course)  were  analyzed.  None  of  the  factors  were  significant. 

3.5  Sickness  and  Perceived  Workload 

Participants’  SSQ  scores  and  perceived  workload  were  both  slightly  higher  with  nVIDIA  than 
with  Miracube.  However,  the  difference  was  not  statistically  significant.  Additionally,  the 
nVIDIA  display  (M=  6.3 1)  resulted  in  higher  workload  scores  than  the  Miracube  display 
(M=  4.44)  for  the  Perceptual  tests. 


4.  Discussion 


In  this  experiment,  we  evaluated  two  SD  systems  in  three  different  types  of  tasks:  Perceptual 
evaluation  with  still  pictures  and  pre-recorded  videos,  live  Robot  Teleoperation,  and  simulated 
indirect  driving  in  Virtual  environments  with  different  types  of  terrain.  Overall,  participants 
completed  their  tasks  significantly  faster  when  they  used  an  SD  in  3-D  mode  compared  to  the 
baseline  2-D  mode.  When  the  effectiveness  of  the  SDs  were  examined  separately,  the  analysis 
showed  that  the  nVIDIA  system  appeared  to  be  more  effective  than  the  Miracube  in  supporting 
faster  response  times  and  course  completion  times. 

For  the  Perceptual  tests,  participants  perceived  hazardous  terrains  in  videos  significantly  faster 
using  nVIDIA  than  Miracube.  Similarly,  for  the  Robot  Teleoperation  tests,  participants 
navigated  the  robot  significantly  faster  using  nVIDIA  than  Miracube,  saving  -15%  of  navigation 
time  per  trial.  Participants  also  operated  the  robot  slightly  faster  with  the  3-D  mode  than  with  the 
2-D  mode,  although  the  3-D/2-D  difference  failed  to  reach  statistical  significance. 

For  the  Floating  course  of  the  Virtual  tests,  the  results  showed  that  participants’  accuracy  was 
significantly  better  with  3-D  than  with  2-D.  In  contrast  to  the  results  of  the  Robot  Teleoperation 
tests,  participants'  accuracy  was  significantly  better  with  Miracube  than  with  nVIDIA.  For  the 
Obstacle  course  of  the  Virtual  tests,  the  results  showed  that  participants  performed  slightly  better 
with  3-D  than  with  2-D,  although  the  difference  failed  to  reach  statistical  significance.  For  the 
Negative  Terrain  course  of  the  Virtual  tests,  none  of  the  factors  were  found  to  be  significant. 

The  lack  of  difference  in  operator  performance  for  the  Miracube  display  may  be  due  to  the  fact 
that  the  Negative  Terrain  tracks  had  a  very  large  amount  of  graphics  that  had  to  be  rendered  (as 
opposed  to  the  Floating  tracks  and  Obstacle  tracks)  and,  for  the  Miracube  display,  two  image 
streams  needed  to  be  rendered  for  the  3-D  condition  but  only  one  image  stream  for  the  2-D 
condition.  Therefore,  images  in  the  3-D  Miracube  condition  could  have  been  slightly  slower  or 
“sluggish”  compared  to  2-D,  and  this  could  account  for  why  3-D  did  not  result  in  better 
perfonnance  than  2-D  for  the  Miracube  display.  For  the  nVIDIA  display,  on  the  other  hand, 
there  was  no  difference  in  the  amount  of  image  streams  that  had  to  be  rendered  in  the  2-D  v  3-D 
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conditions.  In  other  words,  there  was  no  additional  “cost”  associated  with  rendering  3-D  images 
for  the  nVIDIA  display  compared  to  the  2-D  condition.  As  a  result,  participants  drove  faster  in 
the  3-D  condition  (about  10%  reduction  in  course  completion  time)  and  also  more  accurately 
(42%  reduction  in  time  off-course). 

The  data  did  not  indicate  a  significant  effect  of  Display  type  on  sickness  or  workload  for  any  of 
the  tasks.  This  suggests  that  the  two  displays  are  comparable  in  their  workload  and  sickness- 
inducing  characteristics.  Display  type  also  did  not  have  an  effect  on  self-rated  performance  for 
the  Robot  Teleoperation  or  the  Virtual  scenarios.  However,  people  using  the  nVIDIA  display 
thought  they  perfonned  better  on  the  Perceptual  tasks  than  people  using  the  Miracube  display. 
This  could  be  due  to  the  difficulty  of  the  tasks  combined  with  the  viewing-position  constraints  of 
the  Miracube  display  (i.e.,  if  participants  moved  out  of  the  “ideal”  viewing  eye  height,  the  stereo 
was  not  as  clear  [less  L/R  channel  separation,  thus  more  ghosting,  with  consequent  reduction  in 
binocular  depth  perception],  whereas  the  nVIDIA  uses  shutter  glasses,  which  are  not  adversely 
affected  by  changes  in  user  eye  position  relative  to  the  display  screen). 

In  conclusion,  the  results  of  this  study  indicate  that  stereoscopic  displays  do  have  advantages 
over  tradition  2-D  displays  with  respect  to  depth  perception,  specifically  on  driving  and  robot 
teleoperation  tasks.  Furthermore,  the  results  show  that  the  nVIDIA  system  is  a  superior  system 
for  these  tasks  than  the  Miracube  system.  Thus,  it  is  recommended  that  the  U.S.  Anny  employ 
these  systems  in  robot  operator  controls  and  indirect-vision  driving  displays.  This  should  result 
in  better  perfonnance,  and  potentially  less  loss  of  life  and  equipment. 
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Appendix  A.  Demographic  Survey  and  Summary  of  Results 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Participant  # 


Age 


Major 


Date 


Gender 


1.  What  is  the  highest  level  of  education  you  have  had? 

Less  than  4  yrs  of  college _  Completed  4  yrs  of  college _  Other _ 

2.  When  did  you  use  computers  in  your  education?  (Circle  all  that  apply') 

Grade  School  Jr.  High  High  School 

Technical  School  College  Did  Not  Use 

3.  Where  do  you  currently  use  a  computer?  ( Circle  all  that  apply ) 

Home  Work  Library  Other _  Do  Not  Use 

4.  For  each  of  the  following  questions,  circle  the  response  that  best  describes  you. 

How  often  do  you: 

Use  a  mouse?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  a  joystick?  Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

Use  a  touch  screen?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Use  icon-based  programs/software? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 
Use  programs/software  with  pull-down  menus? 

Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 
Use  graphics/drawing  features  in  software  packages? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 
Use  E-mail?  Daily,  Weekly,  Monthly,  Once  every  few  months,  Rarely,  Never 

Operate  a  radio  controlled  vehicle  (car,  boat,  or  plane)? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

Play  computer/video  games? 

Daily,  Weekly,  Monthly,  Once  every  few  months.  Rarely,  Never 

5.  Which  type(s)  of  computer/video  games  do  you  most  often  play  if  you  play  at  least  once  every  few  months? 

6.  Which  of  the  following  best  describes  your  expertise  with  computer?  (check  V  one) 

_ Novice 

_ Good  with  one  type  of  software  package  (such  as  word  processing  or  slides) 

_ Good  with  several  software  packages 

_ Can  program  in  one  language  and  use  several  software  packages 

_ Can  program  in  several  languages  and  use  several  software  packages 

7.  Are  you  in  your  usual  state  of  health  physically?  YES  NO 
If  NO,  please  briefly  explain: 

8.  How  many  hours  of  sleep  did  you  get  last  night? _ hours 

9.  Do  you  have  normal  color  vision?  YES  NO 

10.  Do  you  have  prior  military  service?  YES  NO  If  Yes,  how  long _ 
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Summary  of  Demographic  Survey  Results 


Age:  Ranged  from  18-40 

Mean  =  25.8  Standard  Deviation  =  6.2 

Gender:  19  Males 

13  Females 

How  often  do  you: 

Drive  a  car?  Never  =  1  Rarely  =  0  Once  every  few  months  =  0  Monthly  =  0 
Weekly  =  1  Daily  =  30 


Use  a  joystick?  Never  =  6  Rarely  =17  Once  every  few  months  =  3 
Monthly  =  1  Weekly  =  2  Daily  =  3 

Operate  a  radio-controlled  vehicle?  Never  =15  Rarely  =12 

Once  every  few  months  =  2  Monthly  =  1  Weekly  =  1  Daily  =  1 
Play  computer/video  games?  Never  =  2  Rarely  =  7  Once  every  few  months  =  5 
Monthly  =  5  Weekly  =  4  Daily  =  9 

Which  types  of  computer/video  games  do  you  most  often  play  if  you  play  at  least  once  every  few 
months?  Driving  games  =  5  Other/None  =  27 

Which  of  the  following  best  describes  your  expertise  with  computer/video  games? 

Novice  =  6 

Good  with  one  type  of  software  package  =  8 
Good  with  several  software  packages  =14 

Can  program  in  one  language  and  use  several  software  packages  =  2 

_ Can  program  in  several  languages  and  use  several  software  packages  =  2 _ 

What  type(s)  of  radio-controlled  vehicle  do  you  use  most  often  if  you  use  one  at  least  every  few 
months?  Ground  vehicles  =  6  Other/None  =  26 

Which  of  the  following  best  describes  your  expertise  with  radio-controlled  vehicles? 

Novice  =  26 

Good  with  one  type  of  software  package  =  4 
Good  with  several  software  packages  =  2 

Can  program  in  one  language  and  use  several  software  packages  =  0 
Can  program  in  several  languages  and  use  several  software  packages  =  0 


Are  you  in  your  good/comfortable  state  of  health  physically? 

Yes  =  31 

No  =  1 

Do  you  have  normal/corrected-to-normal  vision?  Yes  =  30 

No  =  2 
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22 


Appendix  B.  NASA-TLX  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Please  rate  your  overall  impression  of  demands  imposed  on  you  during  the  exercise. 


1.  Mental  Demand:  How  much  mental  and  perceptual  activity  was  required  (e.g.,  thinking,  looking,  searching, 
etc.)?  Was  the  task  easy  or  demanding,  simple  or  complex,  exacting  or  forgiving? 

LOW  H-H-l-H-H-l  HIGH 
1  23456789  10 

2.  Physical  Demand:  How  much  physical  activity  was  required  (e.g.,  pushing,  pulling,  turning,  controlling, 
activating,  etc.)?  Was  the  task  easy  or  demanding,  slow  or  brisk,  slack  or  strenuous,  restful  or  laborious? 

LOW  l-l-l-H-l-H-l-l  HIGH 
1  23456789  10 

3.  Temporal  Demand:  How  much  time  pressure  did  you  feel  due  to  the  rate  or  pace  at  which  the  task  or  task 
elements  occurred?  Was  the  pace  slow  and  leisurely  or  rapid  and  frantic? 

low  h-h-I-I-I-H-I  high 

1  23456789  10 

4.  Level  of  Effort:  How  hard  did  you  have  to  work  (mentally  and  physically)  to  accomplish  your  level  of 
performance? 


LOW  H-l-l-l-l-l-l-l-l  high 
1  23456789  10 

5.  Level  of  Frustration:  How  insecure,  discouraged,  irritated,  stressed  and  annoyed  versus  secure,  gratified,  content, 
relaxed  and  complacent  did  you  feel  during  the  task? 

LOW  l-l-l-l-l-l-l-l-l-l  HIGH 

1  23456789  10 

6.  Performance:  How  successful  do  you  think  you  were  in  accomplishing  the  goals  of  the  task  set  by  the 
experimenter  (or  yourself)?  How  satisfied  were  you  with  your  performance  in  accomplishing  these  goals? 

LOW  l-l-l-l-l-l-l-l-l-l  HIGH 

1  23456789  10 


24 


Appendix  C.  Simulator  Sickness  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Instructions:  Please  indicate  how  you  feel  right  now  in  the  following  areas,  by  circling  the 
word  that  applies. 


1. 

General  Discomfort 

None 

Slight 

Moderate 

Severe 

2. 

Fatigue 

None 

Slight 

Moderate 

Severe 

3. 

Headache 

None 

Slight 

Moderate 

Severe 

4. 

Eye  Strain 

None 

Slight 

Moderate 

Severe 

5. 

Difficulty  Focusing 

None 

Slight 

Moderate 

Severe 

6. 

Increased  Salivation 

None 

Slight 

Moderate 

Severe 

7. 

Sweating 

None 

Slight 

Moderate 

Severe 

8. 

Nausea 

None 

Slight 

Moderate 

Severe 

9. 

Difficulty  Concentrating 

None 

Slight 

Moderate 

Severe 

10. 

Fullness  of  Head 

None 

Slight 

Moderate 

Severe 

11. 

Blurred  vision 

None 

Slight 

Moderate 

Severe 

12. 

Dizzy  (Eyes  Open) 

None 

Slight 

Moderate 

Severe 

13. 

Dizzy  (Eyes  Closed) 

None 

Slight 

Moderate 

Severe 

14. 

Vertigo* 

None 

Slight 

Moderate 

Severe 

15. 

Stomach  Awareness** 

None 

Slight 

Moderate 

Severe 

16. 

Burping 

None 

Slight 

Moderate 

Severe 

*  Vertigo  is  a  disordered  state  in  which  the  person  or  his/her  surroundings  seem  to  whirl  dizzily:  giddiness. 

Stomach  awareness  is  usually  used  to  indicate  a  feeling  of  discomfort  which  is  just  short  of  nausea. 

Are  there  any  other  symptoms  you  are  experiencing  right  now?  If  so,  please  describe  the  symptom(s)  and  rate 
its/their  severity  below.  Use  the  other  side  if  necessary. 
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Appendix  D.  Scoring  Procedure  for  the  Simulator  Sickness  Questionnaire 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Symptoms  scored  0  (None)  -  3  (Severe) 

Nausea  (Raw)  -  Sum  of  General  discomfort,  increased  salivation,  sweating,  nausea,  diff 
concentrating,  stomach  awareness,  burping 

Nausea  sub  scale  =  Nausea  (Raw)  x  9.54 

Oculomotor  -  Sum  of  general  discomfort,  fatigue,  headache,  eye  strain,  diff  focusing,  diff 
concentrating,  blurred  vision 

Oculomotor  sub  scale  =  Nausea  (Raw)  x  7.58 

Disorientation  -  Sum  of  diff  focusing,  nausea,  fullness  of  head,  blurred  vision,  dizzy 
(eyes  open),  dizzy  (eyes  closed),  vertigo 

Disorientation  sub  scale  =  Nausea  (Raw)  x  13.92 

TSS  =  [Nausea  (Raw)  +  Oculomotor  (Raw)  +  Disorientation  (Raw)]  x  3.74 
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List  of  Symbols,  Abbreviations  and  Acronyms 


2-D 

two  dimensional 

3-D 

three  dimensional 

ANCOVA 

analysis  of  covariance 

IR 

infrared 

IVD 

indirect-vision  driving 

NASA-TLX 

National  Aeronautics  and  Space  Administration  -  Task  Load  Index 

SD 

stereoscopic  display 

SSQ 

Simulator  Sickness  Questionnaire 

SUGV 

small  unmanned  ground  vehicle 
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